1 link tagged with all of: llama.cpp + gemma + nemotron + small language model
Click any tag below to further narrow down your results
Links
This is NVIDIA's model card for Nemotron-3.5-Bolt-E4B-4GB, a small 4B-effective model post-trained from Gemma-4-E4B-it and aimed at fast, low-memory local voice and device assistants. The page is mostly deployment instructions, covering how to run it through llama.cpp, vLLM, SGLang, Ollama, LM Studio and several agent frameworks.
- Built for low memory and low time-to-first-token, with improved multi-turn tool use and instruction following over its Gemma base.
- Ships as a Q4_K_M quantized GGUF with up to 64K tokens of context, and supports toggling reasoning mode through the chat template.
- An optional DFlash drafter model enables speculative decoding for faster generation, and the model is licensed under OpenMDW-1.1 for commercial or non-commercial use.